Motion detection of audio sources to facilitate reproduction of spatial audio spaces

ABSTRACT

Embodiments of the invention relate generally to electrical and electronic hardware, computer software, wired and wireless network communications, and wearable/mobile computing devices configured to facilitate production and/or reproduction of spatial audio and/or one or more audio spaces. More specifically, disclosed are systems, components and methods to acoustically determine displacements of audios sources (or portions thereof), such as a subset of speaking users, for providing audio spaces and spatial sound field reproduction, for example, for a remote listener. In one embodiment, a media device includes transducers to emit audible acoustic signals into a region including one or more audio sources, acoustic probe transducers configured to emit ultrasonic signals and acoustic sensors configured to sense received ultrasonic signals reflected from an audio source. A controller can determine a displacement of the audio source. Examples of displacement include locomotion, gesture-related motion and orientation changes.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is co-related to U.S. Nonprovisional patent applicationNo. 13/954331 filed Jul. 30, 2013, and entitled “Acoustic Detection ofAudio Sources to Facilitate Reproduction of Spatial Audio Spaces,” whichis herein incorporated by reference in its entirety and for allpurposes.

FIELD

Embodiments of the invention relate generally to electrical andelectronic hardware, computer software, wired and wireless networkcommunications, and wearable/mobile computing devices configured tofacilitate production and/or reproduction of spatial audio and/or one ormore audio spaces. More specifically, disclosed are systems, componentsand methods to acoustically determine displacements of audios sources(or portions thereof), such as a subset of speaking users or listeners,for providing audio spaces and spatial sound field reproduction, forexample, for a remote listener.

BACKGROUND

Reproduction of a three-dimensional (“3D”) sound of a sound field usingloudspeakers is vulnerable to perceptible distortion due to, forexample, spectral coloration and other sound-related phenomena.Conventional devices and techniques to generate three-dimensionalbinaural audio have been generally focused on resolving the issues ofcross-talk between left-channel audio and right-channel audio. Forexample, conventional 3D audio techniques, such as ambiophonics,high-order ambisonics (“HOA”), wavefield synthesis (“WFS”), and thelike, have been developed to address 3D audio generation. However, someof the traditional approaches are suboptimal. For example, some of theabove-described techniques require additions of spectral coloration, theuse of a relatively large number of loudspeakers and/or microphones, andother such limitations. While functional, the traditional devices andsolutions to reproducing three-dimensional binaural audio are notwell-suited for capturing fully the acoustic effects of the environmentassociated with, for example, a remote sound field.

Further, there are drawbacks of using traditional three-dimensionalbinaural audio devices and solutions to reproduce audio originating froman audio source moving within a sound field, and to change thedirectivity of spatial audio responsive to a displacement of the audiosource. One conventional approach, for example, relies on the use ofvideo and/or image detection of the persons to identify audio sources.The capture of images of objects may lead to inadvertent identificationof objects to which spatial audio is to be directed. For example,persons viewable through a conference room window may be detected bytraditional three-dimensional binaural audio devices and solutions as arecipient of audio, while those persons are not intended to be deemedparticipants.

Thus, what is needed is a solution for audio capture and reproductiondevices without the limitations of conventional techniques.

BRIEF DESCRIPTION OF THE DRAWINGS

Various embodiments or examples (“examples”) of the invention aredisclosed in the following detailed description and the accompanyingdrawings:

FIG. 1 illustrates an example of a media device configured to detectdisplacement for facilitating three-dimensional (“3D”) audio spacegeneration and/or reproduction, according to some embodiments;

FIG. 2 illustrates an example of a media device configured to detectdisplacement for modifying directivity of three-dimensional (“3D”) audiospace generation and/or reproduction, according to some embodiments;

FIG. 3 illustrates an example of a media device configured to determinelocomotion of an audio source acoustically to facilitate spatial audiogeneration and/or reproduction, according to some embodiments;

FIG. 4 depicts an example of a media device configured to generatespatial audio based alternative and/or supplemental techniques todetermine locomotion and/or one or more positions of an audio source,according to some embodiments;

FIGS. 5A to 5C depict examples of determining locomotive displacementand displacement related to gestures and/or orientation changes of anobject, according to some embodiments;

FIGS. 6A to 6D depict an environment mapper configured to map positionsof one or more surfaces disposed in a sound field, according to someembodiments;

FIG. 7 depicts an example of a media device configured to generatespatial audio based on ultrasonic probe signals, according to someembodiments;

FIG. 8 depicts a controller including a signal modulator operable togenerate pseudo-random key-based signals, according to some embodiments;

FIG. 9 depicts an example of a gesture detector, according to someembodiments;

FIG. 10 is an example flow of determining displacement of an object in asound field, according to some embodiments;

FIGS. 11A and 11B depict another example of a media device includingcomponents to compensate for an environment in which it is disposed,according to some embodiments; and

FIG. 12 illustrates an exemplary computing platform disposed in a mediadevice in accordance with various embodiments.

DETAILED DESCRIPTION

Various embodiments or examples may be implemented in numerous ways,including as a system, a process, an apparatus, a user interface, or aseries of program instructions on a computer readable medium such as acomputer readable storage medium or a computer network where the programinstructions are sent over optical, electronic, or wirelesscommunication links. In general, operations of disclosed processes maybe performed in an arbitrary order, unless otherwise provided in theclaims.

A detailed description of one or more examples is provided below alongwith accompanying figures. The detailed description is provided inconnection with such examples, but is not limited to any particularexample. The scope is limited only by the claims and numerousalternatives, modifications, and equivalents are encompassed. Numerousspecific details are set forth in the following description in order toprovide a thorough understanding. These details are provided for thepurpose of example and the described techniques may be practicedaccording to the claims without some or all of these specific details.For clarity, technical material that is known in the technical fieldsrelated to the examples has not been described in detail to avoidunnecessarily obscuring the description.

FIG. 1 illustrates an example of a media device configured to detectdisplacement for facilitating three-dimensional (“3D”) audio spacegeneration and/or reproduction, according to some embodiments. Diagram100 depicts a media device 102 configured to receive audio data (e.g.,from a remote source of audio or audio in recorded form stored in areadable media) for presentation as spatial audio to recipient orlistener 140 a. In some examples, at least two transducers, such astransducers 120 a, operating as loudspeakers can generate acousticsignals that can form an impression or a perception at a listener's earsthat sounds are coming from audio sources disposed anywhere in a space(e.g., 2D or 3D space) rather than just from the positions of theloudspeakers. Further, media device 102 can be configured to transmitdata representing the acoustic effects associated with sound field 180.According to various embodiments, sound field 180 can be reproduced so aremote listener (not shown) can perceive the change of positions ororientations of listener 140 a relative, for example, to an audiopresentation device at a remote location (or any other reference, suchas a point in space that coincides with position of audio presentationdevice).

In particular, diagram 100 illustrates a media device 102 configured toat least include one or more transducers 120, one or more acoustictransducers 112 a, 112 b, and 112 c, one or more acoustic sensors 111 a,111 b, and 111 c, a displacement determinator 175, and an interfacecontroller 177. Acoustic transducers 112 are configured to generateacoustic probe signals configured to detect objects or entities, such asaudio sources (e.g., listener/vocal speaker 140 a), in sound field 180.Acoustic sensors 111 are configured to receive the reflected acousticprobe signals for determining the distance between an object that causedreflection of the acoustic probe signal back to media device 102.

Displacement determinator 175 is configured to determine a displacementof at least a portion of an object based on characteristics of thereflected acoustic probe signals (e.g., reflected ultrasonic signals).In some embodiments, displacement determinator 175 is configured todetermine values representative of modified characteristics. Examples ofsuch characteristics include a distance, a direction (e.g., a direction,or an angular direction, such as a vector, defined by an angle relativeto a reference line, such as the face of media device 102), and thelike. Examples of modified characteristics include a variation indistance, a variation in direction, such as a variation in angle (e.g.,changes in angles between two positions), and the like. As such,displacement determinator 175 can use the direction and/or distance ofan object, such as an audio source, to calculate, for example, adisplacement of listener 140 a or a portion thereof. To illustrate,consider that acoustic transducer 112 a generates an acoustic probesignal 130 a to probe the distance to an object, such as listener 140 a.Reflected acoustic probe signal 130 b (or a portion thereof) returns, orsubstantially returns, toward acoustic transducer 112 a where it isreceived by, for example, acoustic sensor 111 a. The distance betweenlistener 140 a and the face of media device 102 can be determined basedon, for example, a time delay between transmission of acoustic probesignal 130 a and reception of reflected acoustic probe signal 130 b.

In some embodiments, one or more acoustic transducers 112 a, 112 b, and112 c can generate unique ultrasonic signals as acoustic probes and emitthe unique ultrasonic signals in directions from which they are emittedbased on the acoustic probe transducer 112 a, 112 b, or 112 c. Forexample, an acoustic probe signal can include data indicating, forexample, the position in space from which it originates (e.g., anidentification of acoustic transducer 112 a based on its predeterminedposition relative to any number of acoustic sensors 111 a, 111 b, and111 c). As such, a surface associated with an object may cause reflectedacoustic probe signal 130 b to arrive at acoustic sensors 111 b and 111c as well as acoustic sensor 111 a. Based on time delays associated withreflected acoustic probe signal 130 b being received by acoustic sensors(e.g., at different distances relative to acoustic transducer 112 a),the distance can be determined as well as orientations of one or moresurfaces associated with audio source 140 a, such as a face of alistener.

Based on the displacement, locomotion of the listener or audio source140 a can be detected, according to some embodiments. As shown, considerthat audio source 140 a traverses from point 177 a in space to point 177b in space. As audio source 140 a moves through sound field 180,acoustic probe signals 130 a, 131 a, and 132 a cause reflected acousticprobe signals 130 b, 131 b, and 132 b to respectively to indicate adistance associated with audio source 140 a as the listener transitspast corresponding acoustic sensor at different points in time. As usedherein, at least in some embodiments, the term “locomotion” can, forexample, describe the movement from a position of a predominant portionof an object (or all of an object), whereby movement causes generaldisplacement from a first place or position to a second place orposition. Responsive to displacement indicative of locomotion, mediadevice 102 (e.g., including a controller thereof) can be configured toidentify an action, and/or cause performance of the action, based on thedisplacement. An example of performing such an action includes changingdirectivity of sound beams configured to provide spatial audio. As such,media device 102 can be configured to direct sound beams 134 a forproviding spatial audio when listener 140 a is at point 177 a, andfurther configured to direct sound beams 134 b to listener 140 a at 177b subsequent to locomotion.

According to some embodiments, displacement of at least a portion of anobject, such as listener or audio source 140 a, can be detected.Consider that detection of the motion of a portion of the object isdesired for various purposes. An example of such a purpose includesimplementation of a detected portion as a “gesture,” which can be usedto provide user input into, or cause modification of operation of, mediadevice 102 or any other device. As shown in diagram 190, a gesture canbe detected by motion of an arm 142, or any other appendage, from afirst position 192 (and/or a first direction) to a second position 194(and/or a second direction). As shown, consider that a portion of audiosource 140 a moves in space (e.g., is transitory) while another portionof audio source 140 a is non-transitory or substantially non-transitory.As portion 142 of audio source 140 a moves within sound field 180, oneor more acoustic probe signals, such as acoustic probe signal 130 a cancause reflected acoustic probe signals 130 b to indicate a distanceand/or direction associated with portion 142 at different points intime. As used herein, at least in some embodiments, the term “gesture”can, for example, describe a movement from a position of, or othermotion associated with, a portion of an object, whereby movement causesgeneral displacement from a first position to a second position (e.g.,motion detected by changes in position, as detected by ultrasonicacoustic probe or other types of probes, typically relative to a subsetof acoustic probe signals associated with one or more surfaces that arerelatively stationary or non-transient). Responsive to displacementsindicative of a gesture, media device (e.g., including a controllerthereof) can be configured to identify an action, and/or causeperformance of the action, based on the displacement indicative. Anexample of performing such an action includes modification of theoperation of media device 102. For example, a detected gesture can betranslated into a command or data representing a command as a controlaction. Such a control action can be transmitted to an interfacecontroller 177, which controls inputs and outputs for media device 102.A gesture can translate into input control data 171, which representsreceived user input, to control operation of media device (e.g., turnmedia device off, connect a call, modify selection of audio, etc.).Further, a gesture can translate into output control data 173, whichrepresents output directed to a user, such as changes in volume or audiofeedback as to operation of media device 102, and the like. As audiosource 140 a can cause input into media device 102 via gestures, audiofeedback can be directed only to audio source 140 a (optional) toindicate a state of media device 102 (e.g., the selection of new audiois accepted). Note that arrangements of acoustic transducers and/oracoustic sensors need not be disposed in media device 102, according tosome embodiments. The functionality of a controller and acoustictransducers and/or acoustic sensors can be implemented in a separatedevice that communicates displacement data and position data to mediadevice 102.

FIG. 2 illustrates an example of a media device configured to detectdisplacement for modifying directivity of three-dimensional (“3D”) audiospace generation and/or reproduction, according to some embodiments.Diagram 200 illustrates a media device 202 configured to at leastinclude one or more transducers 220, a controller 270, a displacementdeterminator 275, and various other components (not shown), such as acommunications module for communicating, Wi-Fi signals, Bluetooth®signals, or the like. Media device 202 is configured to receive audiovia microphones 210 (e.g., binaural audio) and to produce audio signalsand waveforms to produce sound that can be perceived by a listener 240.As shown in diagram 200, controller 270 includes a spatial audiogenerator 272. In various embodiments, spatial audio generator 272 isconfigured to generate 2D or 3D spatial audio locally, such as at audiospace 242 a and/or at audio space 242 c, and/or reproduce sound field280 for presentation to a remote listener 294 as a reproduced soundfield 280 a. Sound field 280, for example, can include an audio space242 a and an audio space 242 c.

According to some embodiments, audio space 242 c is formed by modifyingthe directivity of sound beams 231 and 233 (collectively 230 a) used todirect spatial audio to form an audio space 242 a for listener 240 a bydirecting sound beams 231 and 233 to form sound beams 230 c. As such,spatial audio generator 272 can cause the directivity of audio space 242c to track listener 240 a (at a first position) as the listenertraverses in sound field 280 from point (“A”) 277 a to point (“B”) 277b, and from point (“B”) 277 b to point (“C”) 277 c. Listener 240 a isdepicted in FIG. 2 as listener 240 c (at a second position) aftertraversing from point 277 a to point 277 c. Displacement determinator275 can be configured to determine a displacement of listener 240 a todetermine a position of listener 240 c at point 277 c, based onreflected acoustic probe signals 330 b and the like. Reflected acousticprobe 330 b, which can be received into acoustic sensor 111 a, is shownto originate as acoustic probe signal 330 a from an acoustic transducer112 a.

Spatial audio generator 272 is configured to receive audio, for example,originating from remote listener 294 (or from a media storing theaudio), to generate 2D or 3D spatial audio 230 a for transmission tolistener 240 a. In some embodiments, transducers 220 can generate firstsound beam 231 and second sound beam 233 for propagation to the left earand the right ear, respectively, of listener 240 a. Therefore, soundbeams 231 and 233 are generated to form an audio space 242 a (e.g., abinaural audio space) in which listener 240 a perceives the audio asspatial audio 230 a.

According to various embodiments, spatial audio generator 272 cangenerate spatial audio 230 a using a subset of spatial audio generationtechniques that implement digital signal processors, digital filters,and the like to provide perceptible cues for listener 240 a to correlatespatial audio 230 a with a perceived position at which the audio sourceoriginates. In some embodiments, spatial audio generator 272 isconfigured to implement a crosstalk cancellation filter (andcorresponding filter parameters), or variant thereof, as disclosed inpublished international patent application WO2012/036912A1, whichdescribes an approach to producing cross-talk cancellation filters tofacilitate three-dimensional binaural audio reproduction. In someexamples, spatial audio generator 272 includes one or more digitalprocessors and/or one or more digital filters configured to implement aBACCH® digital filter, an audio technology developed by PrincetonUniversity of Princeton, N.J. In some examples, spatial audio generator272 includes one or more digital processors and/or one or more digitalfilters configured to implement LiveAudio® as developed by AliphCom ofSan Francisco, Calif.

Transducers 220 cooperate electrically with other components of mediadevice 202, including spatial audio generator 272, to steer or otherwisedirect sound beams 231 and 233 to a point in space at which listener 240a resides and/or at which audio space 242 a is to be formed. In someembodiments, transducers 220 a are sufficient to implement a leftloudspeaker and a right loudspeaker to direct sound beam 231 and soundbeam 233, respectively, to listener 240 a. Further, additionaltransducers 220 b can be implemented along with transducers 220 a toform arrays or groups of any number of transducers operable asloudspeakers, whereby groups of transducers need not be aligned in rowsand columns and can be arranged and sized differently, according to someembodiments. Transducers 220 can be directed by spatial audio generator272 to steer or otherwise direct sound beams 231 to specific position orpoint in space within sound field 280 to form an audio space 242 aincident with the location of listener 240 a relative to the location ofmedia device 202.

According to various other examples, media device 202 and transducers220 can be configured to generate spatial audio for any number of audiospaces, such as spatial audio 230 a and 230 c directed to form audiospace 242 a and audio space 242 c, respectively, which can include alistener traversing from point 277 a to point 277 c. In someembodiments, spatial audio generator 272 can be configured to generatespatial audio to be perceived at one or more audio spaces 242 a and 242c. For example, remote listener 294 can transmit audio that is presentedas spatial audio 230 a directed to only audio space 242 a, whereby otherlisteners cannot perceive audio 230 a as transducers 220 do notpropagate audio 230 a to other positions, unless listener 240 a moves tothat new position. Note that while listeners 240 a are 240 c aredescribed as such (i.e., listeners), such listeners 240 a and 240 c eachcan be audio sources, too, and can represent the same audio source atdifferent times after locomotion. In some cases, objects 240 a and 240 cneed not be animate, but can be audio devices.

Displacement determinator 2754 is configured to determine approximatepositions, and variations therefrom, of one or more listeners 240 and/orone or more audio spaces 242. By determining approximate displacement ofa listeners 240, spatial audio generator 272 can enhance the auditoryexperience (e.g., perceived spatial audio) of the listeners by adjustingoperation of the one or more crosstalk filters and/or by more accuratelysteering or directing certain sound beams to the listener as thelistener, for example, moves around a room including sound field 280. Inone implementation, displacement determinator 274 uses informationdescribing the approximate variations in positions of audio spaces 242located within sound field 280 to determine the location of a listener240. According to some embodiments, such information can be used bygenerating acoustic probes that are transmitted into sound field 280from media device 202 to determine relative distances (e.g., magnitudesof vectors) and directions (e.g., angular displacement of vectorsrelative to a reference) of audio sources and other aspects of soundfield 280, including the dimensions of a room and the like. Examples ofacoustic probes and other acoustic-based techniques for determiningdirections and distances of audio spaces are described hereinafter.Controller 270 provides distances (and variations thereof) anddirections (and variations thereof) to spatial audio generator 272 tomodify operation of, for example, a cross-talk filer (e.g., angles ordirections from speakers to each of a listener's ears) and/or steerabletransducers to track directivity of spatial audio toward a listener ashe or she moves through sound field 280.

Diagram 200 further depicts media device 202 in communication via one ormore networks 284 with a remote audio presentation device 290 at aremote region. Controller 270 can be configured to transmit audio data203 from media device 202 to remote audio system 290. In someembodiments, audio data 203 includes audio as received by one or moremicrophones 210 and control data that includes information describinghow to form a reproduce sound field 280 a. Remote audio system 290 canuse the control data to reproduce sound field 280 by generating soundbeams 235 a and 235 b for the right ear and left ear, respectively, ofremote listener 294. For example, the control data may includeparameters to adjust a crosstalk filter, including but not limited todistances from one or more transducers to an approximate point in spacein which a listener's ear is disposed, calculated pressure to be sensedat a listener's ear, time delays, filter coefficients, parameters and/orcoefficients for one or more transformation matrices, and various otherparameters. The remote listener may perceive audio generated by listener240 a as originating from different positions of audio spaces 242 a to242 c relative to, for example, a point in space coinciding with thelocation of the remote audio system 290. In particular, the remotelistener can perceive audio sources moving relative to audiopresentation device 290 in reproduced sound field 280 a.

In some cases, remote audio system 290 includes logic, structures and/orfunctionality similar to that of spatial audio generator 272 of mediadevice 202. But in some cases, remote audio system 290 need not includea spatial audio generator. As such, spatial audio generator 272 cangenerate spatial audio that can be perceived by remote listener 294regardless of whether remote audio system 290 includes a spatial audiogenerator. In particular, remote audio system 290, which can providebinaural audio, can use audio data 203 to produce spatial binaural audiovia, for example, sound beams 235 a and 235 b without a spatial audiogenerator, according to some embodiments.

Further, media device 202 can be configured to receive audio data 201via network 284 from remote audio system 290. Similar to audio data 203,spatial audio generator 272 of media device 202 can generate spatialaudio 230 a and 230 c by receiving audio from remote audio system 290and applying control data to reproduce the sound field associated withthe remote listener 294 for listener 240. A spatial audio generator (notshown) disposed in remote audio system 290 can generate the controldata, which is transmitted as part of audio data 201. In some cases, thespatial audio generator disposed in remote audio system 290 can generatethe spatial audio to be presented to listener 240 a regardless ofwhether media device 202 includes spatial audio generator 272. That is,the spatial audio generator disposed in remote audio system 290 cangenerate the spatial audio in a manner that the spatial effects can beperceived by a listener 240 via any audio presentation system configuredto provide binaural audio.

Examples of components or elements of an implementation of media device200, including those components used to determine proximity of alistener (or audio source), are disclosed in U.S. patent applicationSer. No. 13/831,422, entitled “Proximity-Based Control of MediaDevices,” filed on Mar. 14, 2013 with, which is incorporated herein byreference. In various examples, media device 202 is not limited topresenting audio, but rather can present both visual information,including video (e.g., using a pico-projector digital video projector orthe like) or other forms of imagery along with (e.g., synchronized with)audio. According to at least some embodiments, the term “audio space”can refer to a two- or three-dimensional space in which sounds can beperceived by a listener as 2D or 3D spatial audio. The term “audiospace” can also refer to a two- or three-dimensional space from whichaudio originates, whereby an audio source can be co-located in the audiospace. For example, a listener can perceive spatial audio in an audiospace, and that same audio space (or variant thereof) can be associatedwith audio generated by the listener, such as during a teleconference.The audio space from which the audio originates can be reproduced at aremote location as part of reproduced sound field 280 a. In some cases,the term “audio space” can be used interchangeably with the term “sweetspot.” In at least one non-limiting implementation, the size of thesweet spot can range from two to four feet in diameter, whereby alistener can vary its position (i.e., the position of the head and/orears) and maintain perception of spatial audio. Various examples ofmicrophones that can be implemented as microphones 210 a to 210 cinclude directional microphones, omni-directional microphones, cardioidmicrophones, Blumlein microphones, ORTF stereo microphones, binauralmicrophones, arrangements of microphones (e.g., similar to Neumann KU100 binaural microphones or the like), and other types of microphones ormicrophone systems.

FIG. 3 illustrates an example of a media device configured to determinelocomotion of an audio source acoustically to facilitate spatial audiogeneration and/or reproduction, according to some embodiments. Diagram300 depicts a media device 302 including a displacement determinator375, one or more acoustic transducers 312, and one or more acousticsensors 311. Acoustic transducers 312 a and 312 b are configured togenerate acoustic probe signals configured to detect locomotion ofobjects, such as a listener 350, in sound field 380. Acoustic sensors311 a and 311 b are configured to receive the reflected acoustic probesignals for determining the distance between the object that causedreflection of the acoustic probe signal back to media device 302.Displacement determinator 375 is configured to determine the variationsof direction and/or distance of object to calculate, for example, aposition of listener 350 at audio space 361 a (e.g., at point 337 inspace) at a first point in time and another position of listener 350 ataudio space 365 a (e.g., at point 339 in space) at a second point intime. To illustrate, consider that acoustic transducer 312 a generatesan acoustic probe signal 330 a to probe the distance to listener 350 ata first point in time. Reflected acoustic probe signal 330 b (or aportion thereof) returns, or substantially returns, toward acoustictransducer 312 a where it is received by, for example, acoustic sensor311 a. A position determinator (not shown) can determine a distance 344a to audio space 361 a (e.g., relative to line 331 coincident with theface of media device 302 or a reference point 333) based on, forexample, the time delay between transmission of acoustic probe signal330 a and reception of reflected acoustic probe signal 330 b. Similarly,the position determinator can determine a distance 340 a to audio space365 a based on, for example, another time delay. Displacementdeterminator 375 is configured to detect locomotion of listener 350 todetermine variations in position (e.g., differences in instantaneousposition) and rates in change in position, whereby the variations andrates of change in position can be calculated for adjusting, forexample, the directivity of spatial audio responsive to the displacement(e.g., the variations in position).

A spatial audio generator (not shown) of media device 302 is configuredto generate spatial audio based on displacement information determinedby displacement determinator 375. Data 303 representing spatial audiocan be transmitted to remote audio system 390 for generating areproduced sound field 390 b for presentation to a remote listener 294.As shown, audio system 390 uses data 303 to form reproduced sound field390 b in which remote listener 294 perceives audio generated by audiosource 350 as originating from a perceived audio source 351 in aposition in perceived audio space 361 b (point A) at the first point intime (e.g., coinciding with point in time in which listener is disposedat point 337 in sound field 380). That is, audio source 350 is perceivedto originate as audio source 351 in audio space 361 b at a distance 344b from point RL, which coincides with that location of remote listener294. Further, audio source 350 can be perceived to originate as audiosource 351 in audio space 365 b at a distance 340 b from point RL afteraudio source 350 transitions from point 337 (at a first point in time)to point 339 (at a second point in time) in sound field 380.

View 392 depicts a top view of the perceived positions A and C at whichperceived audio sources 351 are displaced respectively at audio spaces361 b and 365 b relative to point RL (and/or reference line 395). Asshown, audio system 390 a generates a perceived audio space 365 b atpoint C at a distance 398 from audio system 390 a in a direction basedon an angle 391 b from a line orthogonal to the face of audio system 390a. Remote listener 294, therefore, perceives audio source 350 of audiospace 365 a (in sound field 380) as being at point C in a direction 393from point RL. Direction 393 can be determined by an angle 391 arelative to line 395. According to some embodiments, a distance can beexpressed as a “radius,” and a direction can be expressed as an “angle,”whereby the distance and direction can be described by in a polarcoordinate system, or any other coordinate system.

FIG. 4 depicts an example of a media device configured to generatespatial audio based alternative and/or supplemental techniques todetermine locomotion and/or one or more positions of an audio source,according to some embodiments. Diagram 400 depicts a media device 402including a position determinator 474 configured to determine a positionof an audio (or sound) source relative to media device 402. Displacementdeterminator 475 can be configured to determine variations of position(i.e., motion or movement) of a portion of an audio source. For example,an audio source may be moving an appendage (i.e., a portion of the audiosource) to provide a “gesture” with which to modify operation of mediadevice 402. Further, displacement determinator 475 can be configured todetermine variations of position of a predominant portion (or allportions) of an audio source. For example, displacement determinator 475can detect locomotion of an audio source transiting from a firstlocation (“1”) 482 a to a second location (“2”) 482 b.

Media device 402 can also include one or more components to determinepositions and/or displacements of objects either instead of the useacoustic probe signals or in combination thereof to supplementdeterminations of positions and/or displacements. As shown, diagram 400is a top view of a media device 402 an array of microphones as an arrayof components 183, each microphone being configured to detect or pick-upsounds originating at a location. Position determinator 474 can beconfigured to receive acoustic signals from each of the microphones ordirections from which a sound, such as a vocalized speech sound,originates. For example, a first microphone can be configured to receivesound 484 a originating from a sound source at location (“1”) 482 a,whereas a second microphone can be configured to receive sound 484 boriginating from a sound source at location (“2”) 482 b. For example,position determinator 474 and/or displacement determinator 475 can beconfigured to determine the relative intensities or amplitudes of thesounds received by a subset of microphones and identify the position(e.g., direction or an angle at which the sounds originate) of an audiosource based on a corresponding microphone receiving, for example, thegreatest amplitude. In some cases, a position can be determined inthree-dimensional space. Position determinator 474 and/or displacementdeterminator 475 can be configured to calculate the delays of a soundreceived among a subset of microphones relative to each other todetermine a point (or an approximate point) from which the soundoriginates, as well as displacements from the point a particular pointin time. Delays can represent farther distances a sound travels beforebeing received by a microphone. By comparing delays and determining themagnitudes of such delays, in, for example, an array of transducersoperable as microphones 183, the approximate point from which the soundoriginates can be determined. In some embodiments, position determinator474 and/or displacement determinator 475 can be configured to determinethe source of sound by using known time-of-flight and/or triangulationtechniques and/or algorithms.

According to some embodiments, displacement determinator 475 can beconfigured to use audio received from one or more microphones 483 todetermine approximate changes in positions at which audio spacestraverse within the sound field. For example, acoustic energy (e.g.,vocalized speech) originating from audio source 485 generally is ofgreater amplitude received into a microphone receiving sound 484 a,which is at a relatively shorter distance to audio source 485, ratherthan, for example, the amplitude and time delays associated with theacoustic energy received at a microphone receiving the sound 484 b.Also, data representing vocal patterns (e.g., as “speech fingerprints”)can be stored in memory (not shown) to be used to match against thoseindividuals who may be speaking in the sound field. An individual whosespeech patterns match that of the vocal patterns in memory then can beassociated with a certain position or audio space. Therefore,displacement determinator 475 can track changes in position of anidentified audio source 485 based on detection of different directionsfrom which the associated vocal patterns originate (e.g., as determinedby identifying a microphone from the group of microphones 483 having agreatest amplitude of audio).

In some embodiments, components 483 can be implemented as antennaeconfigured to receive RF signals, whereby position determinator 474and/or displacement determinator 475 can be configured to use the delaysand/or intensities of RF signals to determine a proximity or positionfor audio source 485. Also, media device 402 can detect varioustransmissions of electromagnetic waves (e.g., radio frequency (“RF”)signals) to determine the relative direction and/or distance of alistener carrying or using a device having a radio, for example, such asa mobile phone. In some cases, the RF signals can be characterized andmatched against RF signal signatures (e.g., stored in memory) toidentify specific users or listeners (e.g., for purposes of generatingindividualized audio). In some examples, one or more image capturedevices (e.g., configured to capture one or more images in visiblelight, thermal RF imaging, etc.) can be used to detect audio sources fordetermining locomotion of a listener.

In alternate implementations, position determinator 474 and/ordisplacement determinator 475 can be configured to receive positioninformation regarding the position of a listener (or audio source)wearing a wearable device 491. The wearable device can be configured todetermine a position of the wearer and position location data (e.g., GPSdata, etc.) via any communication channel to media device 202. Anexample of a suitable wearable device, or a variant thereof, isdescribed in U.S. patent application Ser. No. 13/454,040, which isincorporated herein by reference.

FIG. 5A depicts an example of determining locomotive displacement of atleast a portion of an object, according to some embodiments. Diagram 500is a front view of a sound field disposed, for example, in an X-Z plane.Media device 502 of FIG. 5A includes a displacement determinator 575configured to determine variations of position (i.e., motion ormovement) of an audio source or a portion thereof. According to theexample shown, media device 502 is configured to introduce acousticprobe signals 505 into a sound field 501 a to detect, for example, oneor more surfaces associated with an object. As such, a position of anobject can be determined with which displacement determinator 575 isconfigured to detect locomotion of the object, such as an audio source540 (or listener). Locomotion of an object, such as a listener/audiosource, can be determined as the object passes into, and out of,acoustic signals that are projected in a direction (e.g., apredetermined radial direction) from media device 502, for example, orany device external to, but in communication with, media device 502. Forpurposes of illustration, acoustic probe signals 505 that intercept orotherwise reflect from an object of interest, such as audio source 540,are shown in solid points 503 a, 504 a, and the like, whereas acousticprobe signals 505 that not reflected from an object of interest areshown as hollow or unfilled points, such as points 506 b, 506 c, and thelike. Acoustic probe signals that generally do not reflect from theobject are those signals, such as ultrasonic acoustic probe signals,that propagate nearer or farther than the object of interest (e.g., aprobe signal depicted as an unfilled point, such as point 503 b, mayreflect off a wall of a room at a distance farther than that associatedwith position 507 a of an audio source 540).

In this example, locomotion of audio source 540 is described in terms ofvariations or changes in position and/or variations or changes indirection. A change or variation in direction (e.g., relative to areference point 511) can be referred to a change in angle (e.g.,relative to a reference line 509) at which a position of one or moresurfaces of an object varies over time. In some examples, a variation orchange in direction can refer to a variation or change in a lateraldirection (regardless of whether the lateral direction is associatedwith linear or non-linear/arcuate/circumferential motion). A lateraldirection can coincide with motion and/or displacement that can bepredominantly from one side of a sound field to another side (e.g.,substantially associated with or within a plane, such as substantiallyin an X-Z plane). In some examples, lateral directions can refer tovariations in direction (or direction variations), such as variations507 a and 507 b of directions.

Displacement determinator 575 can operate in accordance with thefollowing example. A position 507 a of an object 540 (e.g., an audiosource) can be determined by receiving a subset of reflected acousticprobe signals (e.g., reflected probe signals 503 a, 504 a, 504 b, and506 a) that reflect from one or more surfaces of object 540 at a firstpoint in time. Distances determined from reflected probe signals probesignals 503 a, 504 a, 504 b, and 506 a can indicate that the reflectedsurfaces are at similar distances, or are substantially within a rangeof distances that specify that the reflected surfaces are associatedwith object 540. Acoustic transducers that emit acoustic probe signalsgive rise to reflected signals 503 a, 504 a, 504 b, and 506 a, wherebythe acoustic probe signals are emitted at known directions (e.g., atknown angles relative to reference line 509).

For purposes of discussion, consider that unfilled points may bereflected probe signals probe signals 503 b and 506 b that indicatelarge or infinite distances (or any other distance not associated withan objection of interest). At a second point in time, consider thatreflected acoustic probe signals 503 b, 504 c, and 506 b indicate achange in distance from the large or infinite distances to distancesassociated with the above-mentioned range of distances (e.g., distancesthat related to the object). Also, reflected acoustic probe signals 503a, 504 a, and 506 a indicate a change of distance from theabove-mentioned range of distances to one or more large or infinitedistances. Further, reflected acoustic probe signal 504 b indicates nochange of distance. Responsive to the changes in distances, displacementdeterminator 575 is configured to identify that object 540 is in alocomotive state in which object 540 varies its position to 507 b.

At a third point in time, consider that reflected acoustic probe signals504 c and 504 d are reflected from surface portions B and C,respectively, of object 540. Thus, reflected acoustic probe signals 504c and 504 d specify the range of distances. Further, consider also thatreflected acoustic probe signals 503 c and 506 c are reflected fromsurface portions A and D, respectively, of object 540. Thus, reflectedacoustic probe signals 503 c and 506 c specify the range of distances.Note further, that distances associated with reflected acoustic probesignals 503 b, 504 b, and 506 b at the second point in time transitionsat the third point in time to large or infinite distances. Responsive tothe changes in distances, displacement determinator 575 is configured toidentify that object 540 is in a locomotive state in which object 540varies its position from position 507 b to position 507 c. Inparticular, the direction of object 540 varies from a directionsubstantially parallel to reference line 509 to a first distance 515 a,whereby the first direction 515 a of object 540 can be associated withangle 508 a.

According to some embodiments, displacement determinator 575 can predictthe fourth point in time at which object 540 arrives at position 507 d.From the third to the fourth points in time, the direction of object 540varies from the first direction 515 a to a second direction 515 b. Asshown, second direction 515 b is at another angle 508 b relative toreference line 509. The fourth point of time at which surfaces A, B, C,and D of object 540 are predicted to coincide with a subset of reflectedacoustic probe signals 503 d, 504 d, 504 e, and 506 d can be determined,for example, by one or more of the rates at which surface A traversesthrough reflected acoustic probe signals 503 a, 503 b, and 503 c,surface D traverses through reflected acoustic probe signals 506 a, 506b, and 503 c, and the rates at which surfaces B and C traverses throughpairs of reflected acoustic probe signals 504 a and 504 b, 504 b and 504c, and 504 c and 504 d. According to some embodiments, displacementdeterminator 575 can determine object 540 is in locomotion by detectingsurfaces A, B, C, and D collectively passing through subsets ofreflected acoustic probe signals at the same time (or substantially atthe same time). According to some embodiments, displacement determinator575 also can determine object 540 is in locomotion by detecting surfacesA, B, C, and D passing through subsets of reflected acoustic probesignals at the same distance (or at substantially the same distances)from point 511, or at the same or substantially same rate of change indistance (e.g., there is movement toward or away from media device 502).Therefore, displacement determinator 575 can be configured to predict avariation 507 b of direction at a fourth point in time (e.g., fromdirection 515 a to direction 515 b).

According to some embodiments, displacement determinator 575 can beconfigured to sense a first subset of reflected acoustic probe signals,as ultrasonic signals, from a subset of surfaces associated with anobject 540 at a first time point, and to sense a second subset ofreflected ultrasonic signals with object 540 at a second time point.Displacement determinator 575 can be configured to calculate one or morevariations of directions between the subset of surfaces at the firsttime point and the subset of surfaces at the second time point to formone or more calculated direction variations, such as direction variation507 a and 507 b. Also, displacement determinator 575 can identify a nextposition of a portion of object 540 (e.g., including some or allportions of object 540) based on the one or more calculated directionvariations that are either in a first lateral direction (e.g., adirection associated with a negative angle) or in a second lateraldirection (e.g., a direction associated with a positive angle) relativeto point 511 in space. In some examples, the one or more variations ofdirections can include one or more angles, whereby the one or morecalculated direction variations can be either in a negative angulardirection (e.g., to the left of reference line 509) or in a positiveangular direction (e.g., to the right of reference line 509) relative topoint 511 in space. In at least one embodiment, locomotion of an audiosource 575 can be determined by movement of each surface of an object(e.g., rather than a subset thereof).

Note that the arrangement, quantity, spacing, patterns, etc. of thedirections of emitted acoustic probe signals and/or reflected acousticprobe signals 505 are not limited to the depiction in diagram 500, aswell as in other diagrams shown herewith. According to variousembodiments, fewer or more reflected acoustic probe signals 505 can beimplemented in a variety of other arrangements, quantities, patterns,and the like. Further, emitted acoustic probe signals can each originatefrom individual acoustic transducers, or a subset of emitted acousticprobe signals can originate from an individual acoustic transducer(e.g., a mechanism can be configured to deflect an emitted acousticprobe signal to different points in a sound field, or an acoustictransducer can be oriented to direct emitted acoustic probe signals todifferent points in space).

FIG. 5B is a front view that depicts an example of detecting gesturesbased on displacement of at least a portion of an object, according tosome embodiments. Diagram 530 is a front view of a sound field in whichacoustic probe signals 505 are emitted into a sound field 501 b.Unfilled points represent emitted acoustic probe signals 505 that arenot reflected, or if so, are associated with a relatively large distance(e.g., distance) or any other distance not associated with an objectionof interest. Media device 502 of FIG. 5B includes a displacementdeterminator 575 configured to determine variations of position (i.e.,motion or movement) of a portion of an audio source 540. In variousembodiments, displacement determinator 575 can be configured to performor otherwise contribute to a determination as to whether a portion ofaudio source 540 is associated with a gesture (e.g., based on variationsof the position audio source 540).

According to some embodiments, a gesture can be determined based on asubset of reflected acoustic probe signals that is associated withvarying characteristics relative to another subset of reflected acousticprobe signals that is associated with non-varying characteristics. Toillustrate, consider that an object, such as audio source 540, islocated at position 507 a of FIG. 5B. In this example, displacementdeterminator 575 can be configured to determine that at least one subsetof reflected acoustic probe signals 503 a, 504 a, 504 b, and 506 aindicate that corresponding surfaces of object 540 are non-transitory(“N”). According to some embodiments, acoustic probe signals that areassociated with non-transitory (“N”) surfaces can indicate that surfacesof an object are substantially non-transitory (e.g., within a range ofmotion that are non-transitory). Further, displacement determinator 575can be configured to determine that at least another subset of reflectedacoustic probe signals are associated with transitory one or moresurfaces. In some embodiments, the term “non-transitory” can refer toone or more one or more surfaces that generally remain stationaryrelative to, for example, a reference point 581.

To illustrate operation of displacement determinator 575, consider thatdisplacement determinator 575 can be configured to determine thatreflected acoustic probe signals 503 a, 504 a, 504 b, and 506 a indicatesurfaces of object 540 are non-transitory (“N”) and that reflectedacoustic probe signals 503 b, 504 b, 506 c, and 506 b are transitory(“T”). In this example, reflected acoustic probe signals 503 b, 504 b,506 c, and 506 b indicate that one or more surface of an object, such asaudio source, are in motion (e.g., relative to other portions of theobject). As shown, a position of a participant in a teleconference, suchas audio source 540, can be detected in a sound field relative to mediadevice 502. The body of the participant can be determined to benon-transitory based on reflected acoustic probe signals 503 a, 504 a,504 b, and 506 a. An appendage or other body portions of the participantcan be in motion relative to the body. As shown, reflected acousticprobe signals 503 b, 504 b, 506 c, and 506 b can vary between a distanceassociated with the body of the participant and other distances (e.g.,an infinite distance). As such, displacement determinator 575 candetermine that the displacement of an arm of a participant can beassociated with a pattern of motion that can be a “gesture,” at least insome cases. Such gestures can indicate a participant's desire to modifyoperation of media device 502.

FIG. 5C is a top view that depicts an example of detecting gesturesbased on displacement of at least a portion of an object, according tosome embodiments. Diagram 560 is a top view of a sound field in whichemitted and reflected acoustic probe signals 505 associated with a soundfield 501 c. In the example shown, a portion of a participant istransitory relative to another portion of an object, such as audiosource 540 of FIG. 5C. Media device 502 of FIG. 5B includes adisplacement determinator 575 configured to determine variations ofposition (i.e., motion or movement) of a portion of an audio source 540.

In various embodiments, displacement determinator 575 can be configuredto perform or otherwise contribute to a determination as to whether aportion of audio source 540 is associated with a gesture (e.g., based onvariations of the position audio source 540). As shown, audio source 540is disposed at position 507 c at a first point in time. In this example,the participant is moving its arm 543 between at least two positions 596and 596. Arm 543 of the participant is shown to move in the X-Y plane ina back-and-forth motion, whereby a distance between arm 543 and mediadevice 502 varies. At a first point in time, an acoustic transduceremits an acoustic probe signal, which is reflected from arm 543 (atposition 594) as reflected acoustic probe signal 552. At a second pointin time, the acoustic transducer emits an acoustic probe signal, whichis reflected from arm 543 (at position 596) as reflected acoustic probesignal 554. The variation in position of the one or more surfaces of arm543 over various points of time. A gesture detector (not shown) can beconfigured to determine whether the variations of position of theportion of the object 540 constitute a gesture.

FIG. 5C also depicts locomotion, and/or changes in orientation, asdetected by a displacement. As shown, displacement determinator 575 candetermine the changes of position of audio source 540 from position 507c to position 507 d. Further, displacement determinator 575 can beconfigured to changes in orientation of an object, such as a change inorientation of a body, face or other portion of a participant. In theexample shown, emitted and/or reflected acoustic probe signals 523 canbe used to determine that reflected acoustic probe signal 524 a variesless in distance than reflected acoustic probe signal 524 b. Asreflected acoustic probe signals 523 are generally in a range ofdistances indicating that one or more surfaces are associated with anobject, variations of distances determined by reflected acoustic probesignals 524 a and 524 b can be used to determine that audio source 540has changed an orientation by an angle 521.

In some embodiments, displacement determinator 575 can include a Dopplerdetector 576 to determine distances and variations thereof. Dopplerdetector 576 can be configured to determine Doppler shifts of acousticsignals (e.g., ultrasonic, audible, or any other type of sound wavesignal) to determine distances and rates of change of distances for oneor more surfaces from which acoustic signals are reflected. To identifymovement, Doppler detector 576 can determine movement caused by aDoppler shift a emitted and/or reflected acoustic probe signal. In someembodiments, emitted and/or reflected acoustic probe signals includeemitted and/or reflected ultrasonic acoustic signals modulated, forexample, in accordance with phase-shifted key (“PSK”) modulation. Forexample, PSK-modulated acoustic probe signals can be modulated withunique pseudo-random sequences for one or more individual PSK signalstransmitted for a corresponding ultrasonic transducer. Doppler shifts ofPSK-modulated signals can be determined to identify movements andvariations of position to determine either locomotion or gestures, orboth. Note that variations in motion, such as gestures, can bedetermined by motion in any number of planes, such as motion (e.g.,rotating motion) that varies in each of the X, Y, and Z planes. Further,the unique probe signals (e.g., unique PSK signals) can also includeidentifiers that indicate which media device the signal originated from,the particular acoustic transducer from which it was emitted (whichenables a media device to derive geometric information), a time at whichthe PSK signal was transmitted, and any other information to determinetime of flight, and other signal characteristics for determiningdistances and/or directions.

FIGS. 6A to 6D depict an environment mapper configured to map positionsof one or more surfaces disposed in a sound field, according to someembodiments. Diagram 600 of FIG. 6A depicts an environment mapper 633configured to map or otherwise determine spatial characteristics ofsurfaces in which a media device or an array of acoustic transducers(which can be disposed in a housing separate from the media device) aredisposed. By determining the spatial characteristics of surfaces in anenvironment (or sound field), a media device can be configured toprovide audio that is compensated for environmental influences, such asthe dimensions and relative positions of the surfaces of a room. In someembodiments, environment mapper 633 can be configured to detectrelatively immobile surfaces, such as a ceiling, a wall, a window, orthe like. As such, environment mapper 633 can be configured to determinespatial characteristics as shown in diagram 600. For example, considerthat an acoustic transducer 609 a emits an acoustic probe signal 601that impinges upon a surface 602 (e.g., surface 602 a, 602 b, or othersurfaces). Environment mapper 633 can determine, for example, an angleat which acoustic probe signal 601 is emitted. In one case, acoustictransducer 609 a can emit acoustic probe signal 601 at an angle 603relative to a line 608 b coincident with a surface of a media device(e.g., a front surface) or with an arrangement of acoustic sensors 607(e.g., acoustic sensors 607 a, 607 b, 607 c, and 607 d). In thisconfiguration, a reflected acoustic probe signal can be received intoacoustic sensor 607 b when surface 602 is parallel (e.g., surface 602a), as an example, with surface 608 b. Further, a distance 606 c can bedetermined by the reflected acoustic probe signal 601 received intoacoustic sensor 607 b, based on time delays, among other signalcharacteristics.

In another case, acoustic transducer 609 a can emit acoustic probesignal 601 at any other angle 605 with a line 608 a coincident withanother surface that can include a front surface of a media device oranother arrangement of acoustic sensors (not shown). Next, consider thatreflected surface is surface 602 b. Therefore, acoustic probe signal 601can reflect from surface 602 b toward acoustic sensor 607 c, which islocated at a distance 606 b from acoustic transducer 609 a, and sensor607 d. The delays between reflected acoustic probe signals 604 b and 604c can be used to determine the general distance 606 c and/or orientationof surface 602. As another example, reflected acoustic probe signal 604a can be received into sensor 607 a at a certain point in time. Based onthe time at which the acoustic probe signal 601 was emitted and receivedinto a specific sensor 607 a (among others), and based on distance 606a, the distance to surface 602 a and orientation of the media device (orthe orientation of the arrangement of sensors 607) can be determined.Note that acoustic transducer 609 a can be configured to determine adistance to, or a spatial characteristic, of an obstruction, such as aceiling, if oriented in accordance with a coordinate system 699 b, or afront wall, if oriented in accordance with a coordinate system 699 a.

In some embodiments, environment mapper 633 can be configured to operateat different times during operation of a media device. For example,environment mapper 633 can operate at start-up (or when power isapplied) to determine the spatial characteristics of a room. As anotherexample, environment mapper 633 can operate at different times duringoperation of a media device, such as periodically or responsive to oneor more events. An example of such an event is movement or motion of amedia device. For instance, if a user re-orientates the media device (orarrangement of acoustic transducers and sensors) based on motion sensordata, such as accelerometer data, environment mapper 633 can operate tore-map or re-characterize the dimensions of an environment, such as theroom dimensions. Environment mapper 633 can also determine spatialcharacteristics (e.g., in terms of direction, distance, etc.) for anyobject, such a wall, an audio source, furniture, etc.

A spatial audio generator (not shown) can implement the values of thedistances and other spatial characteristics, as determined byenvironment mapper 633, to adjust the generation of spatial audio tocompensate for the environment. As environment mapper 633 can mapacoustic paths (e.g., using ultrasonic signals), it can determine pathsof audible audio signals. Therefore, audible spatial audio signals canbe adjusted to compensate for surfaces of the environment that mayotherwise cause reflections therefrom, as characterized by environmentmapper 633. The spatial audio generator can compensate for reflectionsof audible audio from surfaces, such as walls, to optimize formation ofaudio spaces that provide 2D or 3D spatial audio.

FIG. 6B depicts an example of an environment characterized by anenvironment mapper, according to some embodiments. In some embodiments,a media device 612 can include one or more arrangements of acoustictransducers and acoustic sensors. A subset of an arrangement 618 ofacoustic transducers can be configured to emit acoustic probe signals611 to surfaces 613, which can reflect the acoustic probe signal asreflected acoustic probe signal 614 c. In one implementation, anacoustic transducer 619 can be configured or otherwise oriented todirect an acoustic probe signal 611 a to detect surfaces of room 615 atangle relative to other signals generated by one or more other acoustictransducers. For example, acoustic probe signal 611 a can be reflectedas reflected acoustic probe signal 614 a or reflected acoustic probesignal 614 b. In this example, acoustic probe signal 611 a candetermine, for example, one or two difference surfaces from whichreflections propagate. As shown, acoustic probe signal 611 a isreflected to the left of the emitting acoustic transducer.

FIG. 6C depicts another example of an environment characterized by are-oriented environment mapper, according to some embodiments. In theexample shown, acoustic transducer 629 can emit a probe signal 621 athat is reflected back as reflected probe signal 624 a, which reflectsback to media device 622 to the right of acoustic transducer 629. Also,probe signal 621 b can reflect back as reflected probe signal 624 b,which reflects back to media device 622 to the left of the acoustictransducer that generated probe signal 621 b. Based on the geometries ofthe paths over with the acoustic probes propagate, an environment mappercan determine orientation of media device 622 and/or the dimensions ofroom 625.

FIG. 6D depicts another example of an arrangement of acoustictransducers, according to some embodiments. In the example shown, anarrangement 638 of acoustic transducers can be configured to generateprobe signals at different angles (e.g., relative to a line orthogonalto a surface of media device 632). In this case, acoustic transducer 639can emit a probe signal that is reflected back as reflected probe signalto the left of acoustic transducer 639. According to variousembodiments, acoustic transducers and/or acoustic sensors can beoriented in any direction to transmit and/or receive acoustic probesignals.

FIG. 7 depicts an example of a media device configured to generatespatial audio based on ultrasonic probe signals, according to someembodiments. Diagram 700 depicts a media device 701 including a housing703, one or more microphones (“Mic”) 710, one or more ultrasonic sensors(“sensor”) 711, one or more transducers, such as loudspeakers(“Speaker”) 720, and one or more acoustic probe transducers, such asultrasonic transducers 712. Diagram 700 is intended to depict componentsschematically in which acoustic signals (“IN”) enter media device 701,whereas other components are associated with acoustic signals (“OUT”)that exit media device 701. Depicted locations of microphones 710,sensors 711, speakers 720, and transducers 712 are explanation purposesand do not limit their placement in housing 703. Thus, loudspeakers 720are configured to emit audible acoustic signals into a region externalto housing 701, whereas acoustic probe transducers can be configured toemit ultrasonic signals external to housing 701 to detect a position (ora variation thereof) of, or a distance (or a variation thereof) to, oneor more audio sources, such as listeners. Controller 730 can beconfigured to determine a position of at least one audio source, such asa listener, in a sound field, based on one or more reflected acousticprobe signals received by one or more ultrasonic sensors 711. Further todiagram 700, ultrasonic transducer 712 can be driven by driver (“D”) 735that can be modulated by signal modulator 732. In some embodiments,ultrasonic transducer 712 is a piezoelectric transducer.

As shown further in diagram 700, controller 730 includes a signalmodulator 732, a signal detector 734, a spatial audio generator 738, aposition determinator 736, and a displacement determinator 775. Signalmodulator 732 is configured to modulate one or more ultrasonic signalsto form multiple acoustic probe signals for probing distances (and/orlocations) relative to one or more audio sources and/or entities in asound field. In some embodiments, signal modulator 732 is configured togenerate unique modulated ultrasonic signals for transmission fromdifferent ultrasonic transducers 712. Since each unique modulatedultrasonic signal is transmitted from a specific correspondingultrasonic transducer 712, a direction of transmission of the uniquemodulated ultrasonic signal is known based on, for example, theorientation of ultrasonic transducer 712. With a direction generallyknown, the delay in receiving the reflected unique modulated ultrasonicsignal provides a basis from which to determine a distance. Signaldetector 734 is configured to identify one or more reflected modulatedultrasonic signals 702 received into one or more sensors 711. In someembodiments, signal detector 734 is configured to monitor multiplemodulated ultrasonic signals 707 (e.g., concurrently) to isolate varioustemporal and spatial responses to facilitate determination of one ormore positions of one or more audio sources.

Position determinator 736 can be configured to determine a position ofan audio source and/or an entity in the sound field by, for example,first detecting a particular modulated ultrasonic signal having aparticular direction, and then calculating a distance to the audiosource or entity based on calculated delay. Spatial audio generator 738is configured to generate spatial audio based on, for example, audioreceived from microphones 710 for transmission as audio data 746, whichis destined for presentation at a remote audio system. Further, spatialaudio generator 738 can receive audio data 748 from a remote location(or a recorded medium, such as a DVD, etc.) that represent spatial audiofor presentation to a local sound field. As such, spatial audio can betransmitted via speakers 720 (e.g., arrays of transducers, such as thoseformed in a phase-arrayed transducer arrangements) to generate soundbeams for creating spatial audio and one or more audio spaces.

In some examples, spatial audio generator 738 may optionally include asound field (“SF”) generator 737 and/or a sound field (“SF”) reproducer739. Sound field generator 737 can generate spatial audio based on audioreceived from microphones 710, whereby the spatial audio is transmittedas audio data 746 to a remote location. Sound field reproducer 739 canreceive audio data 748, which can include control data (e.g., includingspatial filter parameters for a cross-talk cancellation filter and othercircuitry), for converting audio received from a remote location or arecorded medium into spatial audio for transmission through speakers 720to local listeners. Regardless, audio data representing spatial audiooriginating from remote location can be combined at controller 730 withmodulated ultrasonic signals for transmission over at least a portion747 of a common, shared path.

Displacement determinator 775 can be configured to use the directionand/or distance of an object, as determined by position determinator736, to calculate, for example, a displacement of a listener or aportion thereof. Displacement determinator 775 can be configured tocalculate one or more variations of directions between subsets ofsurfaces at a first time point and subsets of surfaces at a second timepoint to identify one or more calculated direction variations. Also,displacement determinator 775 can predict a next position of a portionof an object based on the one or more calculated direction variations.In some examples, the one or more variations of directions can includeone or more angles, whereby the one or more calculated directionvariations can be described as angular directions.

In view of the foregoing, the functions and/or structures of mediadevice 701, as well as its components, can facilitate the determinationof positions and displacements of audio sources (e.g., listeners) usingacoustic techniques, thereby effectively employing acoustic-relatedcomponents to determine movements of listeners in a sound field,including gestures that be interpreted into user input to modifyoperation of media device 701 (e.g., to change volume, to begin or toend a teleconference, to change music selections, etc.).

In some embodiments, media device 701 can be in communication (e.g.,wired or wirelessly) with a mobile device, such as a mobile phone orcomputing device. In some cases, such a mobile device, or any networkedcomputing device (not shown) in communication with media device 701, canprovide at least some of the structures and/or functions of any of thefeatures described herein. As depicted in FIG. 7 and subsequent figures(or preceding figures), the structures and/or functions of any of theabove-described features can be implemented in software, hardware,firmware, circuitry, or any combination thereof. Note that thestructures and constituent elements above, as well as theirfunctionality, may be aggregated or combined with one or more otherstructures or elements. Alternatively, the elements and theirfunctionality may be subdivided into constituent sub-elements, if any.As software, at least some of the above-described techniques may beimplemented using various types of programming or formatting languages,frameworks, syntax, applications, protocols, objects, or techniques. Forexample, at least one of the elements depicted in FIG. 7 (or any figure)can represent one or more algorithms. Or, at least one of the elementscan represent a portion of logic including a portion of hardwareconfigured to provide constituent structures and/or functionalities.

For example, controller 730 and any of its one or more components, suchas signal modulator 732, signal detector 734, spatial audio generator738, position determinator 736, and displacement determinator 775 can beimplemented in one or more computing devices (i.e., any audio-producingdevice, such as desktop audio system (e.g., a Jambox® implementingLiveAudio® or a variant thereof), mobile computing device, such as awearable device or mobile phone (whether worn or carried), that includeone or more processors configured to execute one or more algorithms inmemory. Thus, at least some of the elements in FIG. 7 (or any figure)can represent one or more algorithms. Or, at least one of the elementscan represent a portion of logic including a portion of hardwareconfigured to provide constituent structures and/or functionalities.These can be varied and are not limited to the examples or descriptionsprovided.

As hardware and/or firmware, the above-described structures andtechniques can be implemented using various types of programming orintegrated circuit design languages, including hardware descriptionlanguages, such as any register transfer language (“RTL”) configured todesign field-programmable gate arrays (“FPGAs”), application-specificintegrated circuits (“ASICs”), multi-chip modules, or any other type ofintegrated circuit. For example, controller 730 and any of its one ormore components, such as signal modulator 732, signal detector 734,spatial audio generator 738, position determinator 736, and displacementdeterminator 775 can be implemented in one or more computing devicesthat include one or more circuits. Thus, at least one of the elements inFIG. 7 (or any figure) can represent one or more components of hardware.Or, at least one of the elements can represent a portion of logicincluding a portion of circuit configured to provide constituentstructures and/or functionalities.

According to some embodiments, the term “circuit” can refer, forexample, to any system including a number of components through whichcurrent flows to perform one or more functions, the components includingdiscrete and complex components. Examples of discrete components includetransistors, resistors, capacitors, inductors, diodes, and the like, andexamples of complex components include memory, processors, analogcircuits, digital circuits, and the like, including field-programmablegate arrays (“FPGAs”), application-specific integrated circuits(“ASICs”). Therefore, a circuit can include a system of electroniccomponents and logic components (e.g., logic configured to executeinstructions, such that a group of executable instructions of analgorithm, for example, and, thus, is a component of a circuit).According to some embodiments, the term “module” can refer, for example,to an algorithm or a portion thereof, and/or logic implemented in eitherhardware circuitry or software, or a combination thereof (i.e., a modulecan be implemented as a circuit). In some embodiments, algorithms and/orthe memory in which the algorithms are stored are “components” of acircuit. Thus, the term “circuit” can also refer, for example, to asystem of components, including algorithms. These can be varied and arenot limited to the examples or descriptions provided.

FIG. 8 depicts a controller including a signal modulator operable togenerate pseudo-random key-based signals, according to some embodiments.Controller 830 is shown to include a spatial audio generator 831, asignal modulator 832, a signal detector 834, and a position determinator836. In some embodiments, spatial audio generator 831 provides datarepresenting spatial audio for combination with one or more modulatedultrasonic signals generated by signal modulator 832. In someembodiments, signal modulator 832 is configured to generatephase-shifted key (“PSK”) signals modulated with unique pseudo-randomsequences for one or more individual PSK signals transmitted for acorresponding ultrasonic transducer. Thus, signal modulator 832 cangenerate unique ultrasonic signals, with at least one unique ultrasonicsignal being generated for emission from a corresponding acoustic probetransducer. In some examples, the unique ultrasonic signal is emitted ina direction associated with an orientation of an acoustic probetransducer. The orientation can form a basis from which to determine adirection.

Ultrasonic sensors can sense reflected modulated ultrasonic signals fromone or more surfaces, a subset of the surfaces being associated with anaudio source (e.g., a listener). The reflected unique pseudo-randomsequences for one or more individual PSK signals, depicted as “PSK1,”“PSK2,” . . . , and “PSKn,” can be received from the ultrasonic sensorsand provided to signal detector 834. In some examples, signal detector834 can be tuned (e.g., variably tuned) to different pseudo-randomsequences to provide multiple detection of different pseudo-randomsequences, wherein the detection of pseudo-random sequences of PSK1,PSK2, and PSKn can be in parallel (or in some cases, in series). In someembodiments, signal detector 834 can be configured to operate tomultiply received signals by an expected pseudo-random sequence PSKsignal. An expected pseudo-random sequence for a PSK signal multipliedwith different pseudo-random phase-shift keyed sequences generatewaveforms with an average of zero, thereby making the signal essentiallyzero. However, multiplying the expected pseudo-random sequence PSKsignal by reflected version of itself (e.g., a positive (“+”) valuemultiplied by a positive (“+”) value, or a negative (“−”) valuemultiplied by a negative (“−”) value) generates a relatively strongerresponse signal, whereby the average value is non-zero, or issubstantially non-zero. As such, signal detector 834 may multiply one ormore received waveforms by an expected pseudo random sequence PSK tostrongly isolate the waveform sought.

Position determinator 836 includes a direction determinator 838 anddistance calculator 839. In some examples, direction determinator 838may be configured to determine a direction associated with a particularreceived PSK signal. For example, a specific pseudo-random sequence PSKsignal can originate from a predetermined acoustic probe transducerhaving a specific orientation. Thus, when a pseudo-random sequence for aPSK signal is identified, the corresponding direction (or angle) can bedetermined. Distance calculator 839 can be configured to calculate adistance (or radial distance) to an object that caused reflection of apseudo-random sequence PSK signal. In some examples, a reflection from adistant surface may be equivalent to a delay of the pseudo-randomsequence. Thus, a delay in the multiplied waveform, when compared to theexpected transmitted pseudo-random sequence PSK signal, can beequivalent to isolating reflections at a particular range. Multipleinstances of such multiplications can be performed in parallel. As such,reflections can be detected at multiple distances in parallel. Forexample, multiplications can occur at expected delays at incrementaldistances (e.g., every 6 or 12 inches). A non-zero result determined ata particular delay indicates the range (e.g., 8 feet, 6 inches) from amedia device. Note, too, that echoes not at a selected range incrementmay become invisible or attenuated, thereby improving the response forthe specific one or more ranges selected. This can improve spatial andtemporal resolutions. According to some examples, spatially-separatedultrasonic sensors can provide a slight time difference in the receivedsignal, and, thus can provide orientation information in addition todistance information. Based on the determined direction and distances,position determinator 836 can determine a distance, for example, from apoint in space incident with a local audio system to the audio sourcebased on a sensed reflected ultrasonic signal from surfaces associatedwith an audio source. This information can transmitted as audio data837, which can be used to generate a reproduced sound field to reproducespatial audio at a remote location (or a local location). In someembodiments, the functionality of position determinator can be combinedwith that of signal detector 834.

Displacement determinator 840 includes a direction variation detector842 and a distance variation detector 844. Direction variation detector842 is configured to determine variations of direction (i.e., due tomotion or movement) of an audio source, or a portion thereof, in termsof variations of angles or lateral distances, as described herein.Distance variation detector 844 can be configured to determine distancesand variations thereof based on, for example, Doppler shifts of acousticsignals (e.g., ultrasonic, audible, or any other type of sound wavesignal). Distance variation detector 844 can be configured to determinerates of change of distances for one or more surfaces from whichacoustic signals are reflected. Displacement determinator 840,therefore, can determine variations in distances and directions (e.g.,an angular direction) in any range of motion in an X, Y, and Zcoordinate system (or expressed in polar coordinates). Such motion canbe locomotive or a gesture.

Locomotion detector 846 is configured to detect locomotion or movementof one or more audio sources or objects that traverse one or more pathsis a space associated with a sound field. In at least some embodiments,locomotion detector 846 determines predominant portions (e.g., allportions) of an object that are in motion, as detected by acoustic probesignals. Gesture detector 848 is configured to detect movement of one ormore portions of one or more audio sources or objects, whereby thedetected motion need not be related to locomotion. Displacement of atleast a portion of an object, such as a listener or audio source, can bedetected relative to non-transitory or substantially non-transitoryportions of one or more audio sources. Gesture detector 848 detects thata particular movement associated with a portion of an object matches apattern of motion specifying a gesture. An example of such motion is agesture in which a user moves its hand in a rotational pattern or in anup-and-down pattern, just to name a few. Interface controller 879 isconfigured to receive data representing the identified gesture andtranslate that gesture into control data and/or a command forinterfacing with the media device. For example, such commands caninclude changing volume, music, and other audio functions.

FIG. 9 depicts an example of a gesture detector, according to someembodiments. As shown in diagram 900, gesture detector 950 includes anaudio source detector 951, a non-transient object detector 952, atransient object detector 953, a motion correlator 954, a gestureidentifier 955, a gesture translator 956, and a control action datagenerator 958. According to some embodiments, gesture detector 950 isconfigured to detect a gesture based on a displacement of at least aportion of an object. For example, audio source detector 951 canidentify an object, such as audio source 902. In some embodiments, audiosource detector 951 is configured to detect locomotion or some motion ofobject over a duration in which it is presumed an animate object (e.g.,an object having at least some motion or some movement above a thresholdover the duration) is indicative of person rather than a wall, aceiling, furniture, and the like.

Non-transient object detector 952 can be configured to determine aportion of an audio source that is non-transitory by, for example,determining a first subset of unique ultrasonic signals associated witha portion (e.g., one or more surfaces) of the object that isnon-transitory. As shown, surfaces 904 are indicated as non-transitory.Transient object detector 953 is configured to determine a second subsetof unique ultrasonic signals that are associated with another portion ofthe object, whereby the other portion 905 transits through a region 906(e.g., up and down) in which unique ultrasonic signals, includingultrasonic signal 907, determine one or more surfaces of arm 905 istransitory. Motion correlator 954 is configured to characterize motionof a portion 905 of object 902 to form characterized motion data, whichdescribes the motion of the portion. Gesture identifier 955 isconfigured to compare the characterized motion against data representingpatterns of motion. Each pattern of motion can be associated with agesture. Therefore, a match of the characterized motion to a pattern ofmotion determines a gesture. Gesture translator 956 is configured totranslate the identified gesture to a control action, such as a commandor interface instruction to modify operation of a media device or toperform any other type of action. Control action data generator 958 isconfigured to generate a control action data signal 960 associated withthe control action or command. The control action signal is configuredto cause modified operation of the audio system or any other device.

According to some embodiments, gesture detector 950 or any othercomponent of a displacement determinator can identify a change inorientation of one or more surfaces of an object rather than a gesture.Displacement that is not locomotive in nature and is not identified as agesture can represent a change in orientation. For example, if adisplacement determinator determines that characterized motion does notmatch a pattern of motion associated with a gesture, but such motion isrelatively transitory (e.g., lower frequency of movements relative to aperson's body), then a change in orientation of a surface, such as auser's face, is detected.

FIG. 10 is an example flow of determining displacement of an object in asound field, according to some embodiments. Flow 1000 starts bygenerating at 1002 unique acoustic probe signals (e.g., unique PSKsignals that can be associated to the acoustic transducers from whichthe unique signals were emitted). At 1004, reflected acoustic probesignals are received, each of which can be received by one or moreacoustic sensors disposed at known geometries. A distance and/or adirection is determined at 1006. A determination at 1008 is made whetherto map the environment to identify audio source and/or surfaces of aroom, etc. If so, the spatial characteristics are determined forsurfaces at 1010. If not, flow 1000 moves to 1012 at which a distanceand/or a direction for an audio source is determined (e.g., forindividual or aggregated reflected probe signals). At 1014, motion isdetected.

A determination at 1016 is made whether to locomotion is associated withthe motion. If so, flow 1000 moves to perform 1020 to 1026 and/or 1030to 1036. At 1020, new and absent acoustic probes signals are detectedas, for example, a user passes through beams of acoustic probes wherenew probe distances are determined and other beams of acoustic probesare “left behind” as the trailing edge of the user causes the absence ofsome probes to appear. At 1022, a rate at which new acoustic probes aredetecting a surface and the loss or absence of other acoustic probes canindicate are relative speed of locomotion, whereby new or lost acousticprobes can be predicted at 1024 to anticipate a position with which totrack the user to change directivity of the spatial audio. At 1026, adirection variation can be determined (e.g., a change of direction toanother angle). At 1030, distances provided by acoustic probes signalsare detected as, for example, being associated collectively with anobject. At 1032, a rate at which the acoustic probes are detectingchanges in distance can be calculated to predict a distance at a futuretime point at 1034. At 1036, a distance variation can be determined,after which directivity of spatial audio can be modified to project 3Daudio to another point in space associated with the listener.

If a determination at 1016 is made that motion is not related tolocomotion, flow 1000 moves to 1040 and 1042 at which non-transitoryobject portion and a transitory object portion are respectivelydetermined. At 1044, motion is correlated to one or more patterns ofmotion to identify a gesture at 1046. At 1048, the gesture is translatedinto a command or control action to change operation of a media deviceor any other device. Flow 1000 continues to 1052 at which adetermination is made whether to end or to continue flow 1000.

FIGS. 11A and 11B depict another example of a media device includingcomponents to compensate for an environment in which it is disposed,according to some embodiments. As shown in diagram 1100 of FIG. 11A,media device 1102 is configured to map an environment to identifysurfaces 1104 a to 1104 e, including surfaces of person 1106. As shown,media device 1102 includes a power adjuster 1120, one or moretemperature sensors 1122, one or more motion sensors 1124, and a mediadevice orientation compensator 1126. In some embodiments, power adjuster1120 is configured to selectively increase or decrease power of theemitted acoustic probe signals. As shown, acoustic probe signal 1110 isnot intense enough for reflected acoustic probe signal 1111 to bereceived into an acoustic sensor. With power increased, reflected probesignal 1113 can be sensed by the acoustic sensor to identify fartherdistances. Temperature sensors 1122 detect the temperature of ambientair that can affect the rate at which the acoustic probes propagatethrough air. Acoustic transducers can operate to compensate fordifferent temperatures. Motion sensors 1124, such as accelerometers, candetect a change from a first orientation 1130 of media device to asecond orientation 1132, whereby the resultant orientation is at anangle 1134. Surface position compensator 1126 is configured to predictspatial positions of the surfaces detected based on the previousdeterminations. As shown, diagram 1150 of FIG. 11B depicts a mediadevice 1102 that has been reoriented. Shortly after the change inorientation, surface position compensator 1126 can predict the relativespatial positions of surfaces 1104 a to 1104 d and can predict whichacoustic transducers and sensors can be associated with a correspondingsurface.

FIG. 12 illustrates an exemplary computing platform disposed in a mediadevice in accordance with various embodiments. In some examples,computing platform 1200 may be used to implement computer programs,applications, methods, processes, algorithms, or other software toperform the above-described techniques. Computing platform 1200 includesa bus 1202 or other communication mechanism for communicatinginformation, which interconnects subsystems and devices, such asprocessor 1204, system memory 1206 (e.g., RAM, etc.), storage device1208 (e.g., ROM, etc.), a communication interface 1213 (e.g., anEthernet or wireless controller, a Bluetooth controller, etc.) tofacilitate communications via a port on communication link 1221 tocommunicate, for example, with a computing device, including mobilecomputing and/or communication devices with processors. Processor 1204can be implemented with one or more central processing units (“CPUs”),such as those manufactured by Intel® Corporation, or one or more virtualprocessors, as well as any combination of CPUs and virtual processors.Computing platform 1200 exchanges data representing inputs and outputsvia input-and-output devices 1201, including, but not limited to,keyboards, mice, audio inputs (e.g., speech-to-text devices), userinterfaces, displays, monitors, cursors, touch-sensitive displays, LCDor LED displays, and other I/O-related devices.

According to some examples, computing platform 1200 performs specificoperations by processor 1204 executing one or more sequences of one ormore instructions stored in system memory 1206, and computing platform1200 can be implemented in a client-server arrangement, peer-to-peerarrangement, or as any mobile computing device, including smart phonesand the like. Such instructions or data may be read into system memory1206 from another computer readable medium, such as storage device 1208.In some examples, hard-wired circuitry may be used in place of or incombination with software instructions for implementation. Instructionsmay be embedded in software or firmware. The term “computer readablemedium” refers to any tangible medium that participates in providinginstructions to processor 1204 for execution. Such a medium may takemany forms, including but not limited to, non-volatile media andvolatile media. Non-volatile media includes, for example, optical ormagnetic disks and the like. Volatile media includes dynamic memory,such as system memory 1206.

Common forms of computer readable media includes, for example, floppydisk, flexible disk, hard disk, magnetic tape, any other magneticmedium, CD-ROM, any other optical medium, punch cards, paper tape, anyother physical medium with patterns of holes, RAM, PROM, EPROM,FLASH-EPROM, any other memory chip or cartridge, or any other mediumfrom which a computer can read. Instructions may further be transmittedor received using a transmission medium. The term “transmission medium”may include any tangible or intangible medium that is capable ofstoring, encoding or carrying instructions for execution by the machine,and includes digital or analog communications signals or otherintangible medium to facilitate communication of such instructions.Transmission media includes coaxial cables, copper wire, and fiberoptics, including wires that comprise bus 1202 for transmitting acomputer data signal.

In some examples, execution of the sequences of instructions may beperformed by computing platform 1200. According to some examples,computing platform 1200 can be coupled by communication link 1221 (e.g.,a wired network, such as LAN, PSTN, or any wireless network) to anyother processor to perform the sequence of instructions in coordinationwith (or asynchronous to) one another. Computing platform 1200 maytransmit and receive messages, data, and instructions, including programcode (e.g., application code) through communication link 1221 andcommunication interface 1213. Received program code may be executed byprocessor 1204 as it is received, and/or stored in memory 1206 or othernon-volatile storage for later execution.

In the example shown, system memory 1206 can include various modulesthat include executable instructions to implement functionalitiesdescribed herein. In the example shown, system memory 1206 includes asignal generator module 1260 configured to implement signal generationof a modulated acoustic probe signal. Signal detector module 1262,position determinator module 1264, displacement determinator module1265, a spatial audio generator module 1266, a locomotion detectormodule 1267, and a gesture detector module 1268 each can be configuredto provide one or more functions described herein.

Although the foregoing examples have been described in some detail forpurposes of clarity of understanding, the above-described inventivetechniques are not limited to the details provided. There are manyalternative ways of implementing the above-described inventiontechniques. The disclosed examples are illustrative and not restrictive.

What is claimed:
 1. A method comprising: generating unique ultrasonicsignals, at least a unique ultrasonic signal being generated by emissionfrom a corresponding acoustic probe transducer; emitting the uniqueultrasonic signal in a direction associated with the acoustic probetransducer; sensing reflected ultrasonic signals from one or moresurfaces, a subset of surfaces being associated with an object;identifying a position of the object relative to a point in space as afunction of characteristics of the reflected ultrasonic signals; mappingan environment to detect relatively immobile surfaces within theenvironment in which the object is substantially positioned to modify apower level associated with the unique ultrasonic signal using a poweradjuster in an audio device configured to emit the unique ultrasonicsignal, the power adjuster being configured to modify the power levelbased on a temperature associated with ambient air in the environmentusing a temperature sensor in the audio device; determining adisplacement of at least a portion of the object; identifying an actionresponsive to the displacement; and performing the action to modifyoperation of the audio system.
 2. The method of claim 1, whereinperforming the action comprises: changing directivity of sound beamsconfigured to provide spatial audio to an audio space including arecipient of audio as the object.
 3. The method of claim 1, whereinperforming the action comprises: detecting a gesture based on thedisplacement of the at least a portion of the object; and modifyinggeneration of spatial audio based on the gesture.
 4. The method of claim1, wherein determining the displacement comprises: determining valuesrepresentative of modified characteristics.
 5. The method of claim 4,wherein determining the values representative of the modifiedcharacteristics comprises: detecting a variation in a direction.
 6. Themethod of claim 5, wherein detecting the variation in the directioncomprises: detecting a variation in an angle.
 7. The method of claim 4,wherein determining the values representative of the modifiedcharacteristics comprises: detecting a variation in a distance.
 8. Themethod of claim 1, further comprising: detecting locomotion of theobject based on the displacement.
 9. The method of claim 1, furthercomprising: sensing reflected ultrasonic signals from the subset ofsurfaces associated with the object; modifying, using the poweradjuster, the power level of the unique ultrasonic signal based on anorientation of at least one of the plurality of acoustic probetransducers in the environment relative to the object using a motionsensor in the audio device; calculating one or more variations ofdistances between the point in space and the subset of surface to formone or more calculated distance variations; and identifying a nextposition of the portion of the object based on the one or morecalculated distance variations that is either closer or farther relativeto the point in space.
 10. The method of claim 1, further comprising:sensing a first subset of reflected ultrasonic signals from the subsetof surfaces associated with the object at a first time point; sensing asecond subset of reflected ultrasonic signals from the subset ofsurfaces associated with the object at a second time point; calculatingone or more variations of directions between the subset of surfaces atthe first time point and the subset of surfaces at the second time pointto form one or more calculated direction variations; and identifying anext position of the portion of the object based on the one or morecalculated direction variations that is either in a first lateraldirection or in a second lateral direction relative to the point inspace.
 11. The method of claim 10, wherein the one or more variations ofdirections comprise: one or more angles, wherein the one or morecalculated direction variations is either in a negative angulardirection or in a positive angular direction relative to the point inspace.
 12. The method of claim 1, further comprising: detecting agesture based on the displacement of the at least a portion of theobject, detecting the gesture comprising: determining a first subset ofthe unique ultrasonic signals associated with another portion of theobject; identifying the first subset of the unique ultrasonic signalsindicate that the another portion of the object is non-transitory;determining a second subset of the unique ultrasonic signals associatedwith the portion of the object; and identifying the second subset of theunique ultrasonic signals indicate that the another portion of theobject is transitory.
 13. The method of claim 12, further comprising:characterizing motion of the another portion of the object to formcharacterized motion; comparing the characterized motion against datarepresenting patterns of motion each of which is associated with agesture; and detecting a match of the characterized motion to a patternof motion to determine an identified gesture.
 14. The method of claim13, further comprising: translating the identified gesture to a controlaction; and generating control action signal associated with the controlaction, wherein the control action signal is configured to causemodified operation of the audio system.
 15. The method of claim 12,further comprising: characterizing motion of the another portion of theobject to form characterized motion; comparing the characterized motionagainst data representing patterns of motion each of which is associatedwith a gesture; detecting no match of the characterized motion to thepatterns of motion; and identifying that the characterized motion isindicative of a change in orientation of the object.
 16. An apparatuscomprising: a plurality of transducers configured to emit audibleacoustic signals into a region including one or more audio sources; aplurality of acoustic probe transducers configured to emit ultrasonicsignals, at least a subset of the acoustic probe transducers each isconfigured to emit a unique ultrasonic signal; a power adjusterconfigured to modify a power level of the unique ultrasonic signal basedon a temperature associated with ambient air in an environment in whichthe one or more audio sources are substantially positioned; atemperature sensor coupled with the power adjuster and configured togenerate a signal indicative of the temperature; a plurality of acousticsensors configured to sense received ultrasonic signals reflected fromthe one or more audio sources; a controller configured to determine adisplacement of at least a portion of an audio source of the one or moreaudio sources; and an environmental mapper configured to determinespatial characteristics of relatively immobile surfaces within theenvironment.
 17. The apparatus of claim 16, wherein the controllercomprises: a locomotion detector configured to detect locomotion of theaudio source based on the displacement; and a gesture detectorconfigured to detect a gesture based on the displacement of the portionof the audio source.
 18. The apparatus of claim 17, further comprising:a motion sensor coupled with the power adjuster, the motion sensorincluding accelerometers configured to generate signals indicative of anorientation of at least one of the plurality of acoustic probetransducers in the environment, and wherein the power adjuster beingconfigured to modify the power level of the unique ultrasonic signalbased on the orientation of the at least one of the plurality ofacoustic probe transducers in the environment relative to the one ormore audio sources; a motion correlator configured to correlate thedisplacement with data representing a gesture; a gesture identifierconfigured to identify the gesture; a gesture translator configured totranslate the gesture to a control action command; and a control actiondata generator configured to generate the control action command tocontrol operation of the apparatus.
 19. The apparatus of claim 16,further comprising: a phase-shift key signal modulator configured togenerate the unique ultrasonic signal.